
METHOD AND APPARATUS FOR TACTILE!* RESPONSIVE USER INTERFACE 



o 



5 " 4 



2 FIEIiD OF THE INVENTION 

| 3 The present invention relates to user interface devices and in 

particular to devices providing tactile responsiveness and having 

5 programmable force-position profiles defining tactile 

6 responsiveness in manipulating a cursor on a screen display. 



7 BACKGROUND OF THE INVENTION 

8 In numerous contexts humans perform tasks by interacting with 

9 machines via actuators having knobs, dials or linear actuators. 

10 Such human interaction in many instances becomes conditioned upon 

11 the responsiveness of the actuator. The human operator interacts 

12 in accordance with tactile feedback perceived through contact with 

13 the actuator knobs, dials or handles. 
For example, in video or film editing using systems as 



14 



15 described in U.S. Patent Nos. 4,937,685 and 4,964,004 which are 
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1 incorporated herein by reference, an editor edits video image 

2 information at a console having a plurality of "control wheels" 

3 (i.e. large dials or knobs). The film or video editor controls 

4 operation of a composition system from an operator's console, as 

5 illustrated in Fig. 1, using two sets of controls, one for each 

6 hand, to control the editing process. Each control set includes a 

7 plurality of finger switches or pushbuttons 110 clustered proximate 

8 to a large rotatable control wheel 112, facilitating tactile 

9 operation with minimal hand movement. As the editor is focussing 
J> on at least one video monitor, viewing frames of visual source 
JL' material during the editing function, it is generally the case that 

the operator will acquire a feel for the various controls and 

J? become acclimated to their functionality through tactile feedback 
therefrom, rather than having to look at the control wheel (s) for 

i 

visual feedback. Accordingly, more efficient human interaction 
i€ with, and sensitivity to the composition system is achieved. 
& The control wheels 112 exhibit tactile responsiveness, such as 

18 detents or clicks, as they are rotated. Typically, a full rotation 
of the wheel 112 is correlated to a unit of time, such as one 
second, of viewing the visual source material being edited. A 

21 corresponding number of "frames" of visual source material will be 

22 viewed during such a time period, depending on the medium or type 

23 of source material being edited. It is most desirable that the 

24 number of frames of source material be correlated to the tactile 

25 responsiveness, i.e. number of clicks, of the wheel 112 during 

26 rotation. For instance, film editing involves standardized source 
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1 material of which twenty-four (24) frames are provided per second. 

2 Thus, it is most desirable that in a full rotation of the wheel 112 

3 (presenting one second of source material) , the wheel respond with 

4 twenty-four (24) clicks, each click corresponding to one frame of 

5 the visual source material. 

6 While film editing involves source material having twenty-four 

7 (24) frames per second, other video medium standards require 

8 different frame rates. The frame rate, or number of frames per 
second according to the National Television System Committee (NTSC) 

1® is thirty (30) frames per second, a standard promulgated for 

MLf television video in the United States. Standards such as PAL* and 
SECAM provide for a standard frame rate of twenty-five (25) frames 

1;T per second in England and France respectively. New standards for 



l^jj high definition television specify a frame rate of thirty (3 0) or 

IS? sixty (60) frames per second. 

ifj Differing frame rate standards relating to visual source 

ry 

17 material and the nature of mechanical detents in actuators, 

18 presents the problem that multiple actuators are required to 

19 facilitate correlation between actuator tactile responsiveness and 

20 the various visual source material standards. As illustrated in 

21 Fig. la, actuators known in the art for providing tactile 

22 responsiveness typically incorporate a mechanical" detent mechanism. 

23 A fixed number of clicks is provided by a spring loaded friction 

24 mechanism 111 coacting with a sprocket 113 having a fixed number of 

25 cogs or detents corresponding to the desired number of clicks per 
2 6 revolution. Therefore, an actuator having twenty-four fixed 
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1 detents is required and dedicated for a film editing context, a 

2 thirty detent actuator is required for a NTSC video editing system, 

3 a twenty five detent actuator is required in the PAL or CCAM video 

4 editing context, etc. The plurality of actuators required limits 

5 the flexibility of visual source material composition systems and 

6 significantly increases the complexity, cost and hardware 

7 requirements of a flexible system. 

8 In addition to the lack of flexibility of use of fixed 
jfe mechanical detent actuators, such actuators disadvantageously 

lg become worn and suffer tactile responsiveness degradation over 

U, 8 

lij time* Other mechanically/spring loaded linear or rotary actuators 



1^2: suffer similar deficiencies, 



^ Likewise, other types of actuators or user interface devices 

are known for permitting users to interact with electronic devices, 

1SS such as personal computers. Such user interface devices, like a 

lg| trackball or mouse as disclosed in U.S. Patent No. 4,868,549 ("the 

ru 

17 1 549 patent"), may include tactile responsiveness in the form of 

18 resistance to movement of the device as the device is actuated and 

19 the cursor moves across predetermined areas of the display screen. 

20 in the '54 9 patent a mouse is disclosed which has an 

21 electromagnetic means, in the form of an electromagnet in 

22 conjunction with a magnetic surface or an electromagnetic brake, 

23 which provides resistance to the movement of a "spherical ball 

24 pickup". Feedback or tactile responsiveness is achieved' in one 

25 embodiment by controlling the degree of sliding friction between a 

26 magnetic portion of the mouse and a magnetic pad surface on which 
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1 the mouse must be actuated. Disadvantageously , the magnetic pad 

2 surface is a requirement in such an embodiment, and the friction 

3 forces between the pad and the mouse may be affected in ways that 

4 may not be predictable and might be detrimental to the tactile 

5 responsiveness. 

6 In another embodiment in the '549 patent, an electromagnetic 

7 brake is included and resistance is provided by the brake directly 

8 to the spherical ball or tracking element. The braking mechanism 
9H= is totally self-contained within the mouse eliminating the need for 

fssff 

ICO a magnetic pad surface. However, while the electromagnetic brake 

IE! provides a stopping mechanism, it cannot provide a continuous 

SJ 

lfy torque to the tracking element, i.e. no torque is applied when the 
13 tracking element is stopped. Such a mechanism cannot be used to 

lfjrj change tactile responsiveness, e.g. to decrease resistance, as a 

l|P function of characteristics of a particular screen display. The 

l|3 resistance provided is only opposed to motion and cannot aid motion 

17 by actively driving the ball to facilitate ease of motion in 

18 certain display areas or to keep the cursor off of the boundary of 

19 a restricted display area. 
20 

21 SUMMARY OF THE INVENTION 

2 2 The present invention provides an actuator having 

23 electronically controllable tactile responsiveness which is 

24 flexibly programmable to facilitate provision in a single actuator 
2 5 of torque-position characteristics, such as a selectable number of 
2 6 detents per actuation through its full operative path. In an 
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1 illustrative case of a rotary actuator the present invention 

2 facilitates provision in a single actuator, of torque versus 

3 angular position , characteristics, such as a selectable number of 

4 detents per revolution. 

5 According to the invention, an actuator is in communication 

6 with a servo motor having a position encoder which outputs position 

7 information to a controller that has access to torque-position 

8 relation information. The output of the controller is a digital 
9^ torque signal, in accordance with the torque-position relation 

log information, which is converted to an analog current signal applied 

13 fLJ to the servo motor to generate torque in the servo motor. The 

12^ torque, presenting a tactile response to a human interacting with 

i y 

13^ ! the actuator, is sensed as a detent or a plurality of detents. 

14Q In further accord with the invention, the controller is a 

133 microprocessor which receives position information, from the 

1^3 encoder, through a counter as a position count. Torque-position 

nJ 

17 relation information is stored in microprocessor accessible 

18 firmware as a table containing a series of particular torque values 

19 corresponding to a series of particular position values. The 

20 torque values, output as digital signals and converted by a digital 

21 to analog converter, can be modified in accordance with a plurality 

22 of stored torque versus position tables to facilitate flexible 

23 ■■ programming of various torque profiles. 

24 Features of the invention include the capacity to store and 

25 modify torque profiles and to select one of a predetermined set of 

26 torque profiles to provide an actuator with a desired tactile 
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1 responsiveness. The torque profiles, stored for example, in 

2 electrically erasable programmable" read only memory can be changed 

3 via a computer in communication with the microprocessor. Upon 

4 system power down and subsequent power up, a previously entered 

5 torque profile can be present as a default profile. 

6 In a further embodiment of the invention, a user interface 

7 device, such as a trackball or mouse, is provided and implemented 

8 with programmable tactile responsiveness. In a mouse or trackball 
embodiment, the device includes an interface mechanism comprised of 

lg at least two sets of wheels that move as a spherical ball or 

if: tracking element is actuated by a user. The wheels are aligned on 

= y= 

l|* mutually orthogonal axes and each of at least two sets of wheels 

lS j has a servo motor attached thereto and a position encoder 

ljp associated with each servo motor. Position information from the 

IS position encoder is received by a controller that has access to 

i s 

Ify tactile force information, i.e. torque-display position 

rii 

17 information. 

18 The torque-display position information relates a position or 

19 coordinate of a display entity or cursor on a display screen of an 

20 electronic device to a force or torque applied to the user 

21 interface device, effecting tactile responsiveness of the user 

22 interface device as a function of the display screen on which the 
2 3 display entity or cursor is manipulated. The controller, having 

24 received the display entity or cursor position information from the 

25 position encoders, generates an output which is a digital signal in 

26 accordance with the force-display position relation information. 
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1 The force can be positive or negative, to assist or resist motion. 

2 In a disclosed embodiment, a digital torque signal output in 

3 accordance with torque-display position information is converted 

4 via a digital to analog converter, to an analog current signal 

5 which is applied to servo motors to generate torque in the servo 

6 motors. The torque generated in the servo motors is translated to 

7 the tracking element or ball of the user interface device and 

8 perceived by the user as tactile responsiveness that is a function 
of the position of the cursor on the screen display. 

log Features of the invention include the capability to effect 



lip*! tactile screen boundaries, and "walls" and "troughs" which 

12^ correspond to button bar functions or icon placement on a drag-down 

13^ menu, by increasing and decreasing resistance to further 

14^ manipulation of the interface device by the user, or by aiding 

IS motion of the device. "Bumps" and other textures can be 

SI 

16o implemented on the screen display and tactilely perceived as the 

§y 

17 cursor is moved. Cell boundaries can be defined by hard stops or 

18 "hills" which a cursor will roll off to limit access to screen 

19 areas or otherwise provide an indication of cursor position without 
2 0 requiring the user to look at the screen. Different tactile 
21 response profiles can be stored to correspond to different screen 
2 2 displays and user applications. Physically impaired people can 
2 3 interface to computer applications without the need for sight or 
24 fine motor skills. 

25 
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1 DESCRIPTION OF THE DRAWING 

2 These and other features and advantages of the present 

3 invention will become more apparent in view of the following 

4 detailed description in conjunction with the accompanying drawing , 

5 of which: 

6 Fig. 1 is an illustration of an operator's console for editing 

7 visual source material in a composition system; 

8 Fig. la is a partially broken-away view of an actuator 

9 according to the prior art having mechanical detents ; 

10 p. Fig. 2 is a block diagram of a system for providing 

llJI programmable tactile feedback in an actuator; 

i y 

12 J^j Fig. 3 is a block diagram of a system for providing 

13 K programmable tactile feedback in an actuator, wherein the 
14 O controller comprises a counter, microprocessor and accessible 
15B firmware; 

16p Fig. 3a is an illustrative diagram of an actuator and 

ry 

17' ~ associated function keys for controlling multiple functions and 

18 providing multiple tactile responses in accordance with the 

19 selected function; 

20 Fig. 4 is a block diagram of a system for providing 

21 programmable tactile feedback in an actuator, wherein the system 

22 further includes a tachometer sensing motor actuation to 

23 generate a corresponding actuation in an associated actuator; 

24 Fig. 5A is a view of a prior art mechanical t means for 

25 introducing resistance in an exercise machine; 

2 6 Fig. 5B is a block diagram illustrating implementation of a 
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1 torque controller according to the invention implemented in an 

2 exercise machine; 

3 Fig. 6 is a block diagram of a joystick implementation of an 

4 actuator with electronically controllable tactile responsiveness? 

5 and 

6 F ig- 7 is a perspective view, partially broken away, of a 

7 trackball implementation of a user interface device according to 

8 the invention? 

gjjS ^y^h^ bA *"A =;=, 8 : g : ^r^ vtewsT, respectively, oT^a- 

iS° -wheel assgrnW-^iTTipl <*mgnt-^rf in ^-K^^^^^ i } yjr> B ^ 

ft 1 Fi 9- 9 is a plan view of the wheel assembly of Figs<> 8A and 8B 

f§2 attached to a motor and encoder assembly? 

^ 3 Fig. 10 is a diagrammatic representation of a pair of wheel 
sets having motors and encoders associated therewith, and 

y|5 contacting the tracking element? 

"a 

3 6 Fig. 10A is a diagrammatic representation of a wheel set 

y 

17 disposed for a third axis (z-axis) responsiveness? 

18 Fig. 11 is a diagrammatic representation of a user interface 

19 device according to the invention configured to interface . to a 

20 personal computer? 

21 Fig. 12 is a block diagram of a system according to the 
2 2 invention ? and 

23 Figs. 13A-13D show a user interface screen and profiles for 

24 tactile responsiveness implemented to effect characteristics of the 
2 5 screen o 
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1 DETAILED DESCRI PTION OF AN IIXUSTRATTVB EMBODIMENT 

2 Referring now to Fig. 2, an actuator, such as a rotary 

3 actuator having a control knob 114 is attached via a shaft to a 

4 servo motor 116. In this illustrative embodiment wherein the 

5 actuator is for use in a film/video editing context, the servo 

6 motor is a PMI 12FVS motor. in the present application, as 

7 discussed in greater detail hereinafter, the servo motor is not 

8 used as a motor per se, but rather as a torque controller. The 
l±9 motor never runs at a significant amount of its rated revolutions 
go per minute, but operates normally in this application in a stalled 
r Jl or semi-stalled state. The preferred motor 116 has an installed 
^2 encoder 118. The encoder 118 is a PMI M23, 300 segment modular 
•=i.3 encoder having an index and providing 3 00 cycles per revolution, 
ML4 which results in 1200 waveform edges from index to index. Note 
□L5 that in this illustrative embodiment it is important that the 

•a 

CJ6 encoder be selected to provide a number of edges which is divisible 

ry • - 

17 - by .factors of two, three, five and eight. Thus, position 

18 information can be electronically divided to provide an integer 

19 number of clicks in selectable modes of 24, 25 and 30 positions per 

20 revolution (corresponding to the film/video editing standards of 

21 24, 25 and 30 frames per second or revolution, as discussed 

22 hereinbefore) . 

23 The position information received from the encoder 118 is 

24 processed by a controller 120 so that it represents a positional 

25 count. The controller 120 accesses stored input data 122 in the 
2 6 form of torque-position relation information which correlates a 
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1 received position count with a related torque value. As noted 

2 hereinbefore, the position count, which is a function of encoder 

3 output information, can be derived by electronically dividing 

4 position information provided by the encoder waveform, as desired 

5 into a selected number of positions or position values. The input 

6 data 122 accessed by the controller 120 will have stored torque 

7 values associated with the selected position values as provided in 

8 accordance with the desired torque profile. The controller 120 
hp outputs the torque value as a digital signal which is converted by 

BP a latchable digital to analog converter 12 4 to an analog voltage. 

SI 

^JL As a voltage applied to the motor would result in a proportional 

H ■ 

§y2 motor speed, the analog voltage is related to motor torque by 

ft generating a proportional motor current using a power amplifier 126 

i=4 in conjunction with a motor power supply 128. The torque related 

£5 current is applied to the motor 116 to present the desired torque 

535 which imparts the desired tactile responsiveness to the control 

fU 

17 knob 114. 

18 In an embodiment illustrated in Fig. 3, the controller 120 

19 comprises a counter 13 0 which receives the servo motor position 

20 information from the encoder 118. A microprocessor 132, such as a 

21 Motorola 6809, receives a position count from the counter 130 
2 2 providing an indication of servo motor position relative to the 
2 3 index. The count provided by the counter will increment or 
24 decrement depending on the direction of the change of position of 
2 5 the servo motor. The microprocessor accesses electrically erasable 
2 6 programmable read only memory 134 (EEPR0M) which is programmed with 
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1 one or more tables of torque-position relation information. Each 

2 * table defines a particular torque profile specifying a torque value 

3 corresponding to a particular position count (i.e. knob/ servo motor 

4 position) . 

5 A main application CPU 136 runs an application which requires 

6 and defines particular torque profiles for the actuator 114. The 

7 main application CPU may run an application which defines the 

8 functionality of a control wheel and related function buttons as 

M9 illustrated in Fig. 3a. In this illustrative embodiment the 

~~ 

So control wheel has an outer dial 14 0 which according to the 

fjl application performs a first function having a fixed number of 

f^2 positions, such as selecting one of a plurality of switch settings. 

;13 The application can assign a second function to the same outer 

t|4 dial 140 and provide a profile assigning an alternative 

ML5 responsiveness to the outer dial actuator, such as assigning a 

•=4 ■ 

ULS lever control function having electronically defined stop 

■ y 

17 positions, when a shift key 142 is depressed. An inner control 

18 knob 144 similarly can be assigned a first function and 

19 corresponding torque profile (such as a free running non-detent 

20 scan function) , by the application running on the main application 

21 CPU, and a second (or other) function and corresponding torque 

22 profile (such as a 30 detent per rotation edit mode, as discussed 
2 3 hereinbefore) , which is invoked such as by depressing an alt 

24 key 146. 

25 The main application CPU 136, upon application initialization, 
2 6 down loads the desired torque profiles to the microprocessor 
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1 accessible EEPROM, via an RS-232 serial r or other communication 

2 port. The desired torque profiles reside in EEPROM and are 

3 selectable via the microprocessor for providing the desired torque 

4 at the appropriate actuator position (s) in accordance with the 

5 requirements of the main application. A desired torque profile can 

6 be selected by a user operating the control knob 144 or outer 

7 dial 14 0 actuators, alone* or with other control functions such as 

8 the alt or shift keys, to be responsive in accordance with the 
y9 first or second function. A change in actuator function, and a 
go corresponding change in actuator responsiveness (i.e. torque 
rfl 1 profile) can be effected via selected key strokes, such as a shift 
JE2 key or function key implementation discussed. 



5 W 



The EEPROM resident tables will not change until a new set of 

94 profiles is programmed, i.e down loaded, into the microprocessor 

(35. accessible memory. Thus, when the system is powered down and 

-s: 

36 subsequently powered up, the previously selected torque profile is 

U 

17 resident and available as a default mode for the respective 

18 actuators. 

19 As illustrated in Fig. 4, the selectable torque profiles and 

20 tactile responsiveness of the actuator according to the invention 

21 can be implemented so that a second actuator 150 is responsive to 

22 a first actuator 114 • , operating substantially as discussed 
2 3 hereinbefore. In certain operations it is desirable to have two 
24 actuators working in conjunction according to a common torque 
2 5 profile. In such a case, the servo motor of one actuator can be 
2 6 used to actually drive a second motor, in addition to its function 
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1 as a torque controller. 

2 For instance , it is desirable when editing film, to turn the 

3 first actuator 114 1 to add one or more frames to one end of the 

4 composition material while removing one or the same number of 

5 frames from an opposite end of the composition material controlled 

6 by the second actuator 150. In such a case, rather than trying to 

7 turn the respective control knobs exactly the same amount, it would 

8 be best to have the second actuator 15 0 respond according to the 

Mi, 

£p first actuator 114 1 and its associated torque profile. 

iO As the first actuator 114 1 is manually rotated N clicks as 

III sensed according to its torque profile implemented as discussed 

Li2 hereinbefore with respect to Fig. 3, the encoder 118' and a 

:13 tachometer 152 associated with the first actuator 114 1 indicate the 

□ 

njk direction and speed, respectively, of the first actuator 114 1 to 

~jL5 the microprocessor 132'. The direction and position of the first 

FE6 actuator 114* is received from the encoder 118 1 through the 

17 counter 130 f . The rate of change of position, i.e. velocity, is 

18 indicated by the tachometer 152 as an analog signal, which must be 

19 converted by an analog to digital converter 154 for processing 

20 digitally by the microprocessor 132 f . The microprocessor 132 f , in 

21 accordance with the count received from the first actuator 114* and 

22 a velocity profile, generates a digital signal which is delivered 

23 to the second actuator digital to analog converter 156 and 

24 converted to an analog signal, increasing power .to a second 

25 actuator servo motor 158. The power increase to the second 

26 actuator servo motor 158 results in an actuation of the second 
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4 

5 



1 motor in a direction according to the direction sensed, and 

2 according to an operation directed by the microprocessor. The 
microprocessor monitors a second actuator encoder 160 to read a 
complementary count from the second actuator 150 being driven, and 
monitors a second actuator tachometer 160 to sense a velocity 

6 comparable to that of the first actuator being manually actuated. 

7 When the comparisons indicate that the second actuator is actuated 

8 in accordance with the manual actuation of the first actuator, the 

9 operation is complete. 

O 10 While the implementation of a driven actuator describes a 

r J.l tachometer for determining velocity of the actuators, it will be 

Si _ 

pj- 2 apprecxated that velocity can be derived by the microprocessor 

^13 using a mathematical operation which takes the first derivative of 

QL4 the rate of change of position information, eliminating the need 

t y , 

□L5 for a tachometer. Further , although a motor power supply is 

" ! « 

Q16 indicated in Fig. 4 for each servo motor , it can be appreciated 

" "I 

17 * that a single power supply can be used for both motors. 

18 Although the invention is described herein in the context of 

19 an actuator in a film/video editing context, one of ordinary skill 

20 in the art will appreciate that selectably programmable tactile 

21 responsiveness according to the invention can be provided in many 

22 contexts in which mode selection of tactile responsiveness is 

23 desirable. 

24 While the actuator having electronically controllable tactile 

25 responsiveness is described herein as providing a selectable number 
of detents or clicks per rotation of a control wheel, it can be 



rs 2 



26 



^ 
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1 appreciated that other torque profiles, such as progressively 

2 increasing torque in one direction or another or increasing torque 

3 to a point of a pseudo hard stop, can be achieved according to the 

4 invention by introducing a torque profile which results in ah 

5 appropriate current applied to the servo motor. 

6 Further, although programmable tactile responsiveness is 

7 described in the context of a rotary actuator application, it will 

8 be appreciated that selectable tactile responsiveness can be 
M:9 implemented according to the invention in other applications and 

riO actuator contexts, such as in linear actuator contexts. 

SJ 

11 Referring now to Fig. 5A and 5B, it will be appreciated by 

.12 those of ordinary skill in the art in view of the foregoing, that 

^13 the electronically controllable tactile responsiveness according to 

= 

HI 4 the invention can be implemented in actuators other than knob type 

i « • 

QL5 actuators and in contexts other than video or film editing 

"SI 

QL6 contexts. Various exercise machines have mechanisms for providing 

ru 

17 resistance, such as the mechanism illustrated in Fig. 5A. The 

18 linear motion, of an exerciser pulling alternately on the handles 

19 300, 302 of Fig. 5B is translated and imparts a rotary motion, to a 

20 take-up spool 304 (Fig. 5A and 5B) . In known exercise machines, 

21 resistance is introduced at the take-up spool by tightening a 

22 mechanical/spring mechanism 306 (Fig. 5A) which increases the work 
2 3 required to impart linear motion to the handles 3 00, 3 02. The 

24 system according to the invention and described hereinbefore can be 

25 implemented in such a context by introducing a bidirectional servo- 

26 motor 308 (Fig. 5B) which is adapted to receive bidirectional 
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. 1 torque versus position information in the form of current profiles 

2 resulting in resistance similar to that introduced by the 

3 mechanical means of 306 of Fig. 5A. The current provided by the 

4 torque controller 310 is a function of torque adjust profiles 312 

5 which are selectable/programmable and stored in a manner as 

6 discussed hereinbefore. 

7 Similarly, referring now to Fig. 6, programmable tactile 

8 responsiveness can be implemented in an actuator such as a joystick 
1^9 actuator 400. In such a context, torque profiles are stored in 
g|0 tables within a torque controller 402 in the form of at least two 

••=. 5 

n ll tables for containing profiles to control motors in at: least two 



jf.2 axes. A first servo motor 403 is attached to a sphere 404 to which 



a joystick 4 06 is fixed. The first motor 4 03 is fixed to the 

BL4 sphere 404 to which the joystick is fixed and controls the tactile 

Q5 responsiveness of the joystick 406 as it is linearly actuated in 

-= 

S6 directions indicated by the arrow A-B. The linear motion of the 

fu 

17 joystick in the direction A-B is translated into a rotary motion by 

18 a shaft 4 08 forming an axis about which the joystick 406 rotates in 

19 a limited manner. The torque controller 4 02 contains at least one 

20 profile table that determines the current provided to the first 

21 servo motor 403 and ultimately determines the particular 

22 responsiveness of joystick 406 as it is actuated in directions A-B. 

23 A second servo motor 410 is mounted to a fixed frame or 

24 surface 412 and controls responsiveness of the joystick 406 as it 

25 is aictuated in the direction indicated by arrow C-D. An assembly 

26 comprised of the sphere 404, joystick 406 and first motor 403 is 
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1 capable of limited rotation about an axis formed by a shaft 414 

2 which is connected at a first end to the second motor 410 and at a 

3 second end to a bearing 416. As the joystick 406 is actuated in 

4 the direction C-D, the sphere 404, and first motor 403 to which the 

5 joystick 406 is attached is actuated having a responsiveness as 

6 determined by at least a second profile table stored in the torque 

7 controller 402. 

8 Although the illustrative embodiments of the exercise 
9^ implementation and joystick implementation describe controlled 

1§3 tactile .responsiveness in a single axis and double axis context 

Iftj respectively, it will be appreciated by those of ordinary skill in 

1^1 the art that tactile responsiveness can be implemented in a 

iW plurality of axes greater than. 2. 

B 

ljjjp Furthermore, it will be appreciated by those of ordinary skill 

IS in the art that various mechanisms, such as the spool of the 

SI • • 

HE! exerciser implementation, are useful for translating torque into 

m 

17* linear force and/or linear force into rotational torque, and that 

18 the tables discussed hereinbefore while containing torque versus 

19 position profiles can be programmed to comprise force versus linear 

20 position profiles. 

21 Referring now to Figs. 7-9, a user interface device can be 

22 implemented according to the invention, to include tactile 
2 3 responsiveness as a function of the position of a display entity, 

24 e.g. cursor, on a screen display. In this illustrative embodiment 

25 a trackball 500 is implemented including a plurality of sets of 

26 drive wheels 502 which contact a tracking member or ball 504. As 
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1 will be understood by those of ordinary skill in the art, 

2 manipulation of the tracking member or ball 504 effects 

3 manipulation or movement of a cursor on a screen display (not shown 

4 in Figs. 7-9) . The details of construction and manufacture of a 

5 trackball and/or mouse implementation will be understood by those 

6 of ordinary skill in the art, and therefore will not be presented 

7 here other than to present significant components and their 

8 interrelationships . 

3: In this illustrative embodiment, the tracking member or ball 

¥& is interconnected in the user interface deyice by an 

lial interconnection mechanism comprised of sets of drive wheels. Each 

ifi of the sets of drive wheels 502, best illustrated in Figs. 8A and 

LLH 

!3 8B, is comprised of a hub 506 about which at least one frame 

2?3 structure 508 is configured. The frame (s) 508 have a plurality of 

D 

X5 frame portions each extending longitudinally through a respective 

tS one of a plurality of barrel-shaped gripping members 510. 

17 Preferably, the outside radius of a large portion of the gripping 

18 members 510 is equal to the outside radius of the drive wheels 502. 

19 Two drive wheels are used, offset slightly, to make the contact 

20 with the ball 504 smooth so as to avoid a "bumpy" feeling as the 

21 ball 504 is actuated and in turn actuates the wheels 502. The 
2 2 gripping members are each rotatable around the frame portion that 

23 extends through and supports it. The gripping members 510 are made 

24 of a polymeric material suitable for establishing gripping contact 

25 with the ball 504. In this illustrative embodiment, as illustrated 
26. in Fig. 8A, two frames 508 are configured about the hub 506. The 
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1 gripping members 510 are offset: or staggered in order to compensate 

2 for gaps between gripping members on a respective frame, to 

3 maintain a gripping member in contact with the ball 504 at all 

4 times . 

5 Each pair of frames 508 attached to a common hub 506 and with 

6 associated gripping members 510, constitutes a wheel set that is 

7 attachable, as illustrated in Fig* 9, to a servo motor 512 and 

8 encoder 514 to form a drive/position assembly 516. In this 
9H= illustrative embodiment the drive/position assembly servo motor is 

lQp used actively as a motor. The servo motor may be a Pittman Model 

lly No. 8322 (manufactured by Pittman, Harleysville, PA) , which 

12>y optionally comes with an integrated optical encoder which fulfills 

IT*- the encoder requirement. At least one drive/position assembly 516 
is configured to apply torque and sense position along a respective 

15=: one of mutually orthogonally disposed axes, e.g. an x-axis 

16p corresponding to cursor movement across a display screen, a y-axis 

fO 

17 orthogonally disposed with respect to the x-axis and corresponding 

18 to cursor movement up and down on a screen, and a z-axis 

19 orthogonally disposed with respect to the x and y axes and 
2 0 corresponding to cursor movement in and out of a screen in a three 

21 dimensional configuration. In some instances, as described 

22 hereinafter, a wheel set is attached to a motor without an encoder, 

23 or just a bearing 518 to implement a complementary slave assembly 
2 4 520 when it may not be desirable to include additional servo motors 
25 and/or encoders. The referenced servo motor, without an encoder, 
2 6 may be employed passively as a bearing. 
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1 To implement a two dimensional user interface device, e.g. 

2 trackball or mouse, the tracking element or ball 504 is configured 

3 to have at least two drive/position assemblies 516 positioned with 

4 the gripping members 510 in contact with the ball. As illustrated 

5 in Fig. 10, a first drive/position assembly 516 is positioned with 

6 the gripping members of its wheel set in contact with the ball, and 

7 includes a servo motor and encoder. A first complementary slave 

8 assembly 52 0 is positioned opposed to the first drive/position 
assembly 516 and has gripping members of its wheel set engaging the 

u 

1£jT side of the ball 504 opposite the first drive/position assembly 

H 

1W 516. 

Sj 

12y A second drive/position assembly 516 • is positioned on an axis 

Co 

13~ orthogonal with respect to the first drive/position assembly 516 



s 



l5p and has a servo motor and encoder attached thereto. A second 

151 complementary slave assembly 520 1 is positioned opposed to the 

lP second drive/position assembly 516 f and has gripping members of its 

17 wheel set engaging the side of the ball 504 opposite the second 

18 drive/position assembly 516*. In the illustrative two dimensional 

19 implementation, the complementary slave assemblies include motors 

20 that are slaved to the motors of the drive/position assemblies. 

21 Such slaved motors produce a complementary torque to assist the 

22 drive/position 1 assemblies in applying a more balanced torque to the 

23 ball. It will be appreciated that a less expensive device can be 

24 implemented according to the invention by merely having the wheel 
2 5 sets opposed to the drive/position assemblies configured with a 
2 6 bearing to passively engage the ball. 
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1 As illustrated in Fig. 10A, in implementing a three 

2 dimensional user interface device according to the invention, a 

3 drive/position assembly 516* ' is positioned along a circumference 

4 of the ball 504 such that the orientation of the wheel set is 

5 orthogonally disposed with respect to the orientation of the wheel 

6 sets of the x and y axis assemblies (in Fig. 10A for simplicity 

7 only one wheel set 516 is shown exemplifying the orientation of the 

8 x and y axis assemblies) . The z-axis drive/position assembly 516 1 1 
9M= is preferably configured with a complementary slave assembly 520 11 

10q disposed along an axis that is perpendicular to an axis along which 

llfy the drive/position assembly 516 1 1 is disposed. Although the 

12^1 functionality of a two dimensional implementation is described 

13 ~~p hereinafter for ease of explanation, it will be appreciated that a 
z— axis drive/position assembly and complementary slave assembly can 

ry 

1EH readily be implemented in a user interface device that is 

-a 

'1<£3 responsive in three dimensions. 

ry ■• 

17 Referring now to Figs, 11 and 12 , the drive/position 

18 assemblies 516 and complementary slave assemblies 520 are 

19 configured with the ball 504 (not shown in Figs. 11 and 12) f as a 

20 user interface device 500 in a system that includes an electronic 

21 device or computer system 528 with a screen 530 on which a cursor 

22 is positioned on a screen display or graphical user interface, as 
2 3 known in the art. The computer to which the user interface device 
2 4 500 is connected has an operating system and is capable of running 
2 5 various application programs which result in various screen 
2 6 displays on which the cursor is manipulated using the user 

- 23 - 

DIKE, BR0N STEIN , 
ROBERTS & CUSHMAN 
TEL (617) 523-3400 
FAX (617) 523-6*40 



1 interface device 500. 

2 The user interface device 500 includes at least a first and 

3 second drive/position assembly 516, 516 f each with a servo motor 

4 534 and encoder 53 6 and associated first and second complementary 

5 slave assemblies 520 f 52 0* for respectively sensing y-axis and x- 

6 axis ball movement to be translated into a cursor position on the 

7 display. In this illustrative embodiment, each of the servo motors 

8 in the drive/position assemblies is connected in series with its 

9 respective complementary slave assembly motor which results in the 
10 J=J motor pairs seeing the same current. In the present application 
llftf each servo motor 53 4 is not used as a motor per se, but rather as 
12 RJ a torque controller. The motor never runs at a significant amount 
13 s of its rated revolutions per minute, but operates normally in this 
14 ry application in a stalled or semi-stalled state. The preferred 

F "i 

ISC? motor, as discussed hereinabove, has an installed encoder 53 6. The 

O 

16~ encoder 536 is matched to the motor and the motor application as 

17 appreciated by one of ordinary skill in the art. 

18 The computer or electronic device 528, as known in the art, 

19 is configured to accept an interface board 532 which includes the 

20 mechanisms required to electronically interface the user interface 

21 device 500 to the computer system 528 and display 530. The 

22 interface board 532 is typically configured to reside in an I/O 

23 slot of the computer 528 and includes a microprocessor 538 which 

24 communicates with the computer 528 via a serial communication 

25 channel 540- In the embodiment illustrated in Fig. 11, the 

26 interface board 532 comprises a counter 542 associated with each 
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1 encoder 536. Each counter 542 receives servo motor position 

2 information from the encoder 118. The microprocessor 538 , such as 

3 a Motorola 6809, receives a position count from each counter 542 

4 providing an indication of position of each servo motor relative to 

5 an index. The count provided by the counter will be incremented or 

6 decremented depending on the direction of the change of position of 

7 the servo motor relative to the index, which is indicative of a 

8 change in position of the ball and the cursor on the screen 

9 display. 

10 O The microprocessor 538 accesses torque profile information 

VI 

lirU from a storage mechanism as a function of the coordinate position 

12fLJ indicated via the encoders, i.e. x-axis position and y-axis 

13 H position. The storage mechanism can be internal to the 

14pj microprocessor and/or external in the form of additional torque 

15Ci profile storage 545 (such as EE PROM, ROM, disk, CDROM etc) . The 

16'rt torque profile information provides an indication of a torque or 

17 force to be applied by/to the motor. The torque is a function of 

18 the position of the cursor on the screen and a function of the 

19 particular screen display on which the cursor is being manipulated. 
2 0 As in the embodiments described hereinbefore, the torque 

21 value, in this case a value for each motor or axis, is output from 

22 the storage mechanism as a digital signal which is converted by a 

23 latchable digital to analog converter (D/A) 544 to an analog 

24 voltage. As a voltage applied to the motor would result in a 

25 proportional motor speed, the analog voltage is related to motor 
2 6 torque by generating a proportional motor current using a power 
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1 driver or amplifier 546 (for each motor). The torque related 

2 current is applied to the motor(s) 516, 516 f , 520, 520 f , to present 

3 the desired torque which imparts the desired tactile responsiveness 

4 to the ball 504. 

5 The computer 528 runs an application, or several applications, 

6 which requires and defines particular torque profiles for the user 

7 interface device 500. Each screen display of an application 

8 running on the computer has torque profile information associated 
9 ~ with that particular screen display to effect a corresponding 

10O particular tactile responsiveness for that screen display. The 

llpj torque profile information which is being processed is stored in 

12fy the microprocessor. Additional torque profile information which is 

CO 

13 2 not immediately required for a running screen display can be stored 

14 ry * n eternal memory associated with the nicroprocessor 545. The 

15J1 torque profile information represents a spatial array that 

16M' indicates the relationship of motor currents or torques as a 

17 function of position parameters for each axis present in the 

18 embodiment. In this illustrative embodiment the array must contain 

19 torque information for x and y axis motor pairs as a function of 
2 0 the x and y coordinate position of the cursor on the particular 

21 screen display(s) . 

22 Preferably, a large volume of torque profile information 

23 defining the tactile responsiveness of numerous screen displays of 

24 an application software package or an operating system is stored in 

25 a database associated with a particular application or applications 

26 that run on the computer. As illustrated in Fig. 12, the computer 
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1 typically runs an operating system or main application 550 which is 

2 stored on some external storage medium such as a disk or CD ROM and 

3 paged or transferred to the computer's main memory as the 

4 application code is running. 

5 A database of torque profiles 552, as part of an application 

6 running under the operating system or with the application 550 , 

7 defines the tactile responsiveness of the user interface device 

8 based on the screen displays of the application (s) . The torque 

H9 profile information 552 is accessible to the application (s) or 

o 

M) operating system(s) via the application's application program 

jCjjl interface (API) , as known in the art. The torque profiles relate 

- £ 

2^2 the tactile responsiveness of the user interface device 500 to the 

§ y 

*K3 graphical user interface (s) or screen display (s) of the application 
550 , as respective torque profiles are downloaded or made available 

3B5 to the microprocessor 538 on the interface board 532 to generate 

Q06 the appropriate digital signals in response to the position 

i y 

17 information received from the encoders, as discussed hereinbefore, 

18 A user interface device driver 554 facilitates communication 

19 between the microprocessor 53 8 on the interface board 53 2 for the 

20 user interface device 500, and the host computer 528, The 

21 microprocessor computes coordinates for a change of cursor position 
2 2 by processing the information received from the encoders and 
23 information known about the original position of the cursor as 
2 4 provided by the host computer over the serial channel 540. The 

25 driver communicates the information related to cursor position to 

26 and from the host computer which effects actual positioning of the 
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1 cursor. In the present embodiment, the driver 554 is generic to 

2 the user interface device 500 and is modified slightly from a mouse 

3 or trackball I/O device driver as known in the art, in that the 

4 driver 554, through an interface to the torque profile information 

5 552 and application software 550 coordinates the downloading of 

6 appropriate torque profile information to the microprocessor based 

7 on indications from the application 550 as to the appropriate 

8 torque profile. 

9T Based on the application being run on the host 528, the driver 

IdP 554 running on the host communicates relevant torque profile 

lfij information to the microprocessor 538. The driver also 

M 

12y communicates information to the microprocessor regarding the 

If present position of the cursor on the display screen of the host 

lftji 528. In response to the coordinate information of the cursor on 

l|=! the display screen, the microprocessor 538 generates digital 

.■3=. 

1§M; information corresponding to the appropriate torque relative to the 

17 position of the cursor on the screen, in accordance with the 

18 relevant torque-position profile for that screen display. The D/A 

19 544 for each axis receives the digital torque information and 

20 produces the appropriate analog signal to the power driver (s) 54 6 

21 which generate a current to apply the positive or negative torque 

22 to the motors resulting in the applicable tactile responsiveness of 

23 the ball 504. 

24 When the trackball or mouse is moved to effect a movement of 

25 the cursor on the display screen 53 0, each encoder 53 6 sends 

26 position information to the microprocessor 538. Position 
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1 information in this illustrative embodiment includes an indication 

2 of the direction and number of steps the encoder is changed in 

3 response to actuation of the associated wheel set in contact with 

4 the manually manipulated ball 504. The microprocessor 538 receives 

5 the magnitude and direction information and tracks the position in 

6 the spatial array of relevant torque profile information to 

7 determine the appropriate torque corresponding to the position 

8 information received. The microprocessor 53 8 communicates the 
|f position information to the user interface device driver which 

1€U effects a change in the position of the cursor by communicating 

1W with the computer as is appreciated by those of skill in the art. 

Igj The microprocessor 538 also conveys torque information to the servo 

ljjT motors r via the D/ As and power drivers as described , to effect 

ijjfi appropriate tactile responsiveness based on cursor position within 

1^1 the screen display of the particular application and torque- 

lfe! position information. 

fy 

17 The torque-position information stored and made accessible to 

18 the microprocessor for implementing tactile responsiveness of the 

19 user interface device according to the invention can be used- to 

20 implement various tactile responses as a function of position of 

21 the cursor on the screen display. Boundaries for cursor 
2 2 containment and restricted display areas can be implemented by 
2 3 effecting stops using maximized motor torque. Among other 

24 responsiveness, tactile "hills" and "troughs" can be implemented to 

25 define tactile contours of a graphical user interface such as 

26 illustrated in Figs. 13A-13D. In this particular example of an 
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1 application, a graphical user interface includes a header showing 

2 a command options banner as it appears on the display screen (Figs. 

3 13a and 13B) . 

4 The command options banner is a button bar on which a cursor 

5 is positioned by a user using a user interface device to point and 

6 click to effect certain functionality. The various commands, i.e. 

7 "File," options," "Window," "Help" can be delineated by tactile 

8 boundaries according to the invention, so that the proper 
sH[ positioning of the cursor within an appropriate area to click and 

ICO invoke the command can be easily done and can be tactilely 

IM perceptible. With or without fine motor skills or vision, the user 

12y actuates the user interface device according to the invention and 

13* feels the position of the cursor on the screen display. Tactile 

o * . 

1*= boundaries are programmed, as discussed hereinbefore and as 

lK illustrated in Figs 13C and 13D, so that higher resistance is 

IK perceived at the boundaries with little or no resistance felt when 

IU 

17 the cursor is properly positioned. 

18 Moving the cursor vertically on the screen toward the button 

19 bar the user will perceive neutral or no resistance in the 

20 unrestricted area 560. A sharp increase in torque will be felt as 

21 the lower boundary 562 of the button bar is encountered. When the 

22 cursor is actuated to a position between the lower boundary and an 

23 upper boundary, i.e. in a trough 564, no resistance is perceived. 
2 4 As the upper boundary 566 is approached the torque increases and as 
2 5 the absolute boundary of the screen is encountered increased torque 
2 6- effects a perceptible stop 568. It should be noted that positive 
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1 and negative torques can be generated according to the invention so 

2 that the user interface device includes a tendency to urge the 

3 cursor into a position centered within the boundaries. 

4 Likewise, when the user interface device is actuated to move 

5 the cursor horizontally along the button bar, as illustrated in 

6 Fig* 13 D, boundaries are established that urge the cursor into the 

7 proper position to activate the desired menu selection. 

8 It should be appreciated that in addition to boundaries formed 
by hills and troughs and walls, the tactile responsiveness of the 

1W user interface device according to the invention can be used to 

im implement "texturing" that is tactilely perceptible. Slight 

l|2f increases in torque can be programmed at selected distances with 



1 £ 3 lesser torque therebetween such that a feeling of bumpiness can be 

Q 

lp|| implemented as the cursor is actuated across a screen display. 

IjSj Elasticity, in the form of increasing torque to a point of reverse 
torque, can be implemented to simulate perceived stretching or 

17 resiliency. Furthermore, given the active nature of the torque 

18 assistance capability of the motor (s) in the user interface device 

19 according to the invention, motion assistance can be effected to 

20 make the device roll off of a bump or hill without manual 

21 assistance (such an application is especially useful where a user 

22 may not have fine motor skills) . 

23 The EEPROM resident tables or arrays of torque profile 

24 information will not change until a new set of profiles is 

25 programmed, i.e down loaded, into the microprocessor accessible 
2 6 memory. Thus, when the system is powered down and subsequently 
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1 powered up, the previously selected torque profile is resident and 

2 available as a default mode for the respective actuators, unless a 

3 particular default state is desired and provided. 

4 Although the invention is described hereinbefore with a 

5 singular screen display, it will be appreciated by those of 

6 ordinary skill in the art that the torque position information can 

7 be structured in a manner such that screens can be nested, and 

8 their corresponding profiles nested so that invoking a new screen 

9 from a present screen invokes a corresponding new set of torque 
1® position information. 

li-j It should be appreciated that although tables or arrays of 

ikj torque profile information are discussed in the illustrative 

lgl embodiment herein for relating cursor screen position with tactile 

S3 

l|j responsiveness of the user interface device, torque values may be 

lg calculated "on the fly" for particular screen displays rather than 

lN storing values associated with particular positions. Additionally, 
rather than having the user interface device providing information 
regarding position and changes thereof, torque values may be 

19 associated with particular cursor locations on a particular screen 

20 display such that the screen generates the position information 

21 which is processed according to the invention to provide resultant 

22 tactile responsiveness. 

23 In the embodiment where a slave motor is connected in series, 
2 4 the slave motor will see the same current as the motor with which 
25 it is connected in series. In such a master/slave motor pair, the 
2 6 motors should be virtually identical motors to effect smoothness of 



im 

18 



- 32 - 



DIKE, BROKSTEIR , 
ROBERTS & CUSHMAN 
TEL (617) 523-3400 
FAX (617) 523-6440 



1 rotation of the ball or tracking element. However, in order to 

2 minimize cost of a system according to the invention, it will be 

3 appreciated that it may be preferable to exclude the slave motor in 

4 favor of a passive bearing. 

5 The description of the invention hereinbefore relates to a 

6 trackball, but it will be appreciated that tactile responsiveness 

7 according to the invention can be implemented in various other user 

8 interface devices, including a mouse, joysticks and other devices 
%l requiring actuation and benefitting from the tactile responsiveness 

1( 5 as a function of position* Further, while "a cursor" is 

llyj manipulated in the embodiment described herein, that term is used 

iCl generically to describe something manipulated in a user interface 

13^ of an electronic device, and it should be appreciated that any of 

1*?* various symbols and interface or display resident entities can be 
manipulated with tactile responsiveness, such as cursors, icons, 
windows, menus, or the like. 

i; 

17 While various embodiments of the invention illustrated herein 

18 describe a main CPU to execute an application program requiring and 

19 defining torque profiles for an actuator, and a separate 6809 
2 0 microprocessor implementing firmware specifying torque-position 
21 relationships, one of ordinary skill in the art will appreciate 
2 2 that torque-position relationships can be implemented in the 
2 3 application CPU without the microprocessor or via numerous other 
24 microcontrollers. Further, while it is described that the torque 
2 5 profiles are in EEPROM accessible to the microprocessor it will be 
2 6 appreciated . that the torque profiles can be stored in 
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1 microprocessor resident or other storage means, such as ROM, RAM, 

2 PALs and the like, and accessed accordingly to implement the 

3 desired tactile responsiveness in an actuator. 

4 Although the invention has been shown and described with 

5 respect to exemplary embodiments thereof, various other changes, 

6 additions and omissions in the form and detail thereof may be made 

7 therein without departing from the spirit and scope of the 

8 invention. 
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